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^ ', Abstract 

Methods of increasing linear optical dynamic range of commercial photocamera 
. for optical-digital imaging systems are described. Use of such methods allows to use 

I commercial photocameras for optical measurements. Experimental results are reported. 

O 

1 Introduction 

Increased interest to hybrid optical-digital systems based on "wavefront coding" [1] and 
! "pupil engineering" [2] principles is observed recently. Such systems allow to create devices, 
which combine highly paralleled optical processing and flexibility of digital image processing 
techniques. This conducing improvement of mass and dimensions of such systems and allows 
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Q ' to create devices with unachievable characteristics for pure optics. 

. Imaging scheme of such kind of hybrid systems is changed by inserting synthesized diffrac- 

^ ' tion optical element - kinoform. This enables to perform optical convolution and register 
optically convolved image by digital photosensor. It's notable that all information about 
^ ■ input scene is contented in greyscale levels of the coded image. Non-linear image processing 
' like gamma-correction, image interpolation, and colour scaling can corrupt coded image and 
bring to difficulties in digital deconvolution. That's why linear registration of images by 
photosensor is important. 

In this paper linearization [3,4] and spatially varying pixel exposures [5] techniques are 
applied for increasing of full and linear dynamic ranges of the images obtained from com- 
mercial photocameras. Using special software allows to exploit linearity of camera's sensor 
and apply inexpensive cameras in optical-digital systems were linearity of signal is highly 
important. 

This paper is organised as follows. Camera's radiometric function is provided both for 
DCRAW converter and Canon's conventional converter in subsection 12. dynamic range 
estimation is provided in subsection 12.21 Dark noise and light-depended noise estimation 
is presented in subsections 12.31 and 12.41 respectively. Increasing of camera's images dynamic 
range using Spatially Varying pixel Exposures technique is discussed in section [31 
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2 Commercial photocamera as measuring device 



In this section the hnearization procedure and measuring characteristics of the commercial 
digital camera Canon EOS 400D are described. The radiometric function was measured in 
order to estimate camera's sensor linearity and saturation level. Linear and full dynamic 
range, black level offset (BLO), temporal and spatial noise were evaluated as well. All RAW 
images were processed by the DCRAW [3] converter using "document mode" without colour 
scaling and interpolation. For viewing and analysing of such images it is suitable to use the 
NIP2 [6,7] open source image analyser. 

2.1 Camera's radiometric functions obtained with different con- 
verters 

For the radiometric function to obtain, pictures of the flat field scene in the light of white 
LEDs were taken. Light was passed through ground glasses to remove flat field's non- 
uniformity, images were taken in a laboratory without any lighting and reflections from the 
surrounding surfaces. Pictures were taken with the exposure time from 1/4000 to 10 seconds, 
4 images were taken for each exposure and averaged. ISO setting was the smallest available 
for Canon EOS 400D camera (ISO 100). Pictures were processed by DCRAW converter in 
"document mode" by command dcraw -4 -T -D -v filename. cr2 to produce pure RAW 12-bit 
images. Also images were processed by conventional Canon converter and obtained colour 
images were decomposed to three greyscale images. Dependencies presented in Fig. [T] are for 
blue-filter covered pixels; curves for all colour-filtered pixels differs only by a constant shift 
along the exposure axis. 
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Figure 1: Signals' mean value in DN versus relative exposure: a) for DCRAW processed 
images, b) for images processed by conventional Canon RAW-converter. 

A 64 by 64 pixel area from the centre of the image was used for the analysis for every 
colour-filter type. During processing of obtained data, black level offset (BLO) of 256 DN 
for DCRAW processed images was disclosed and then removed. Canon converter subtracts 
BLO internally. Mean and standard deviation of pixels values were obtained and results are 
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presented in Fig. [T^ for DCRAW converter and in Fig. [T]d for conventional Canon converter. 
Saturation level for the DCRAW converted data is equal 3726 DN, and for Canon converted 
data saturation level is equal to 65535 DN. One can see that DCRAW processed data for 
radiometric function is linear up to saturation level. 



2.2 Camera's dynamic range estimation 

For estimation of the dynamic range's beginning, the minimal SNR was considered as 2. The 
minimal detectable signal corresponds to SNR=2 is equal 4 DN (see Fig. [2^). It is possible to 
find out relative exposure value that corresponds to the minimal signal: it equals to 1.3 ■ 10^^ 
rel. units, as seen from Fig. [T^. 

A linear function was fitted in experimental data of the radiometric function; then signal's 
value corresponding to the end of a linear dynamic range can be estimated as 3066 DN (see 
Fig.[T^) with relative exposure time 1.0 rel. units for this point. Therefore the linear dynamic 
range with DCRAW data processing is 58 dB. To estimate full dynamic range is necessary to 
measure of the maximum saturation signal and minimal detectable signal. Using the same 
calculation procedure described above and SNR as 2, minimal distinguishable signal remains 
4 DN with relative exposure time 1.3-10"^ rel. units. Maximum detectable signal is 3438 DN, 
hence relative exposure value for this point is 1.12 rel. units. Therefore full dynamic range 
can be estimated as 59 dB. 
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Figure 2: Noise versus signal (both in DN): a) for DCRAW converted data, b) for conventional 
Canon converter. 



The same calculations were provided for data converted by Canon conventional converter. 
Using obtained radiometric function presented in Fig. [T]d and taking into account Fig. [2)d, 
it is possible to estimate linear and full dynamic ranges for non-linearized data. As above, 
minimal signal to noise ratio is 2; thus one can estimate minimum signal value as 400 DN 
and corresponding relative exposure value is 8.2 ■ 10~^ rel. units. Maximum value of linear 
signal is estimated by fitting linear function to experimental data (see Fig. [T]d) and it equals 
24454 DN. Corresponding relative exposure value for such signal is 5.0 • 10~^ rel. units. Thus 
linear dynamic range is considered to be 36 dB. 
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As follows from the obtained data, DCRAW converter allowed to use linear dynamic of 
58 dB from 59 dB of the full dynamic range. By contrast, conventional Canon converter can 
provide only 36 dB of the linear dynamic range. 

It is significantly to note that obtained results are only an approximation of the commer- 
cial camera's photo sensor characteristics because of sufficient dispersion of noise character- 
istics between cameras of same model and presence of the on-chip circuitry. It is discussed 
in subsection 12. 5[ 

2.3 Dark noise estimation 

Noise components can be classified in different ways. One type of classifications divides 
the noise components into random (temporal) and pattern (spatial) components. Random 
components include photon shot noise, dark current shot noise, reset noise, and thermal 
noise. Pattern components are amplifier gain non-uniformity, PRNU, dark current non- 
uniformity, and column amplification offset. In the commercial cameras it is difficult to 
distinguish fine structure of noise components because of on-chip noise-cancelling circuitry. 
Hence only general noise components description and estimation is provided such as dark 
and light-depended noise temporal and spatial components. 

2.3.1 Spatial dark noise 

Spatial dark noise can be referred as pixel-to-pixel variation of the dark signal. In the modern 
commercial CMOS cameras it is difficult to measure spatial dark noise because of presence of 
the noise-cancelling on-chip circuitry [8]; more detailed discussion of this question is provided 
in Section [231 

In order to estimate the spatial dark noise, there were taken 64 dark frames with capped 
camera's objective. The ISO speed was 100 and exposure time was 1/32 sec. Then mean 
value and variance along the averaged dark frame were calculated. According to carried out 
measuring, mean value of the averaged dark frame is 256.0 DN and its standard deviation 
is (^dark.spat ~ 0.4 DN. We are emphasizing that such low spatial dark noise is due to the 
on-chip circuitry noise reduction of the Canon's digital camera. 

2.3.2 Temporal dark noise 

Temporal component of the dark noise was also estimated. For such purpose there were 
taken 64 dark frames. Then arrays of pixels were averaged and the RMS noise of each pixel 
was calculated. After such procedure two another arrays are created: the array of pixel's 
mean values Amean and the array of pixel's standard deviations Agt^ (and consequently the 
array of pixel's variations A^ar)- This procedure is analogous to the PixeLink's method [9]. 

To estimate the temporal dark noise quantitatively, the average variation of the A^ar 
need to be calculated and square root is taken. Consequently, the temporal dark noise can 
be evaluated as follows: 
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where M and N is the heigh and width of the dark frame, respectively. To measure deviation 
of dark temporal noise between pixels, one should calculate the standard deviation from Agtd- 
For the digital camera used in this work there were averaged 64 dark frames (ISO speed 
was 100, exposure time 1/32 sec). Thus the adark.temp ~ 1-6 DN, and uncertainness of the 
temporal dark noise is 0.2 DN. 



2.4 Light-depended noise estimation 

The light-depended noise was evaluated as well. There were taken and averaged images of 
the fiat-field scene. The lighting used was matrix of red, green, and blue LEDs driven with 
DC current. ISO setting was ISO 100, the smallest available in the camera. Objective was 
removed in order to achieve fiat-field homogeneity. A 1024 by 1024 pixel area from the centre 
of the image was used for the analysis. 



2.4.1 Spatial light-depended noise 

As a measure of the spatial light-depended noise, photo-response non-uniformity (PRNU) is 
commonly used [9,10]. PRNU is the standard deviation of the fiat-field image with subtracted 
an averaged dark frame. 

There were taken and averaged 64 pictures of the fiat-field scene. Then averaged dark 
frame was subtracted from averaged picture of the flat-field. Obtained picture was decom- 
posed on three images: pixels corresponded to red colour fllters were stored in B^. array, pixels 
corresponded to flrst green colour fllter were stored in Bg array, and pixels corresponded to 
blue colour fllters were stored in i?^ array. Then for each array there were calculated a stan- 
dard deviation divided by the frame mean value. Thus PRNU for each colour component 
was evaluated as follows: 

PRNU = ^}iEMdE!^o/^ (2) 
FrameMean 

According to our measures, PRNU can be estimated as PRNU < 0.5% {crught.spat ~ 12 DN 
and mean value of the signal is 2600). It is needed to mention that such PRNU is the residual 
PRNU after on-chip circuitry noise-reduction that is performed by electronics of the digital 
camera and can not be turned off. 



2.4.2 Temporal light-depended noise 

Temporal light-depended noise is an uncertainness of light's measuring by each pixel, hence 
the calculation procedure is analogous to the procedure for temporal dark noise evaluation 
(see Subsection I2.3.2p . 

The RMS values for each pixel of the flat-fleld scene's image was calculated, forming two 
another arrays: the array of pixel's mean values Amean and the array of pixel's variations 
Ayar- Then obtained array was decomposed accordingly to the light components R, G, and 
B, same as for PRNU estimation. 

Hence temporal light-depended noise was evaluated for each colour channel separately: 
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Thus it can be summarized that temporal hght-depended noise for each colour channel 
can be estimated as aught.temp ~ 14 DN. 

2.5 Results discussion 

It is important to note that presented results are only an approximation of the commercial 
camera's photo sensor characteristics. This is because of on-chip noise reduction circuitry 
and sufficient dispersion of noise characteristics between cameras of same model. 

In scientific grade technical cameras there are calibration methods controlled by user. By 
contrast, in commercial cameras are used their own proprietary noise-cancelling technologies 
that reduces both dark and light-depended noise. As it mentioned in [8], Canon developed 
on-chip technology to reduce fixed-pattern noise based on Correlated Double Sampling [11]. 
First, only the noise is read. Next, it is read in combination with the light signal. When 
the noise component is subtracted from the combined signal, the fixed-pattern noise can be 
eliminated. 

Moreover, light-depended noise also is been suppressed [8] by on-chip circuitry. Such 
method is called complete electronic charge transfer, or complete charge transfer technology. 
Canon designed the photodiode and the signal reader independently to ensure that the sensor 
resets the photodiodes that store electrical charges. By first transferring the residual discharge 
- light and noise signals - left in a photodiode to the corresponding signal reader, the sensor 
resets the diode while reading and holding the initial noise data. After the optical signal and 
noise data have been read together, the initial noise data is used to remove the remaining 
noise from the photodiode and suppress random noise. Thus only an estimation of the 
commercial camera's noise characteristics is possible. 

3 Further increasing of the camera's dynamic range 

Besides linearization, several techniques such as spatially varying pixel exposures [5] (SVE) 
and Assorted Pixels [12] can be applied for further increase of camera's dynamic range. The 
key idea of such techniques is to use data from colour filters on photosensor to estimate true 
value of oversaturated pixels (FigjS]). 

To reconstruct a linear HDR image from the oversaturated one, several steps should be 
performed. Such steps are the calibration, construction of the SVE-image, and linearization 
of the constructed image. First, the camera's response function to the desirable lightsource is 
obtained. During this calibration process, the camera's response function to the lightsource 
as well as correction coefficients for SVE-images linearization are calculated. Secondly, an 
oversaturated image is analysed and saturated pixels are replaced using information from 
the neighbour pixels. Such constructed SVE-image is characterised by a 7-like non-linearity. 
Finally, constructed SVE-image is linearized using correction coefficients, which were ob- 
tained on the first (calibration) step. Obtained linear HDR image is characterised by a broad 
dynamic range and a linear response to the registered light. 
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Figure 3: The dynamic range of an image detector can be improved by assigning different 
exposures to pixels. 

It is necessary to estimate the amount of quantization levels that can be obtained using 
SVE technique. If total number quantization levels is q and the number of different exposures 
in the pattern is then total oi q - K quantization levels lie within the range of measurable 
radiance values. Only the quantization levels contributed by the highest exposure within 
any given overlap region. Total number of unique quantization levels according to [5] can be 
determined as: 



where R(x) rounds-off x to the closest integer. Hence using Eq. [Hone can estimate quantiza- 
tion levels for Canon EOS 400D commercial photo camera used in this work. Using DCRAW 
converter, it is possible to obtain 3066 linear quantization levels for each pixel. Assume 
that ei is transmittance of red light, 62 is transmittance of green light, and 63 of blue light. 
Experimentally for A = 0.63/im He-Ne laser radiation were obtained that 62/61 = 0.2 and 
63/62 = 0.45. Substituting this into Eq. IH Q ~ 7200 quantization levels can be obtained, 
which is a considerable improvement over q = 3066 for the same image detector. 

4 Conclusion 

The use of linearized RAW data from the commercial photo camera and its adoption for 
measurements is presented. Using DCRAW converter with proper parameters, it is possible to 
exploit linearity of camera's photo sensor response to registered light and use the commercial 
camera as a measuring device. 

For Canon EOS 400D digital camera used in this work with DCRAW converted data 
were obtained the following characteristics: linear dynamic range is 58 dB and full dynamic 
range 59 dB with 12bit ADC. Spatial dark noise is (Jdark.spat ~ 0.4 DN, and bias is 256.0 DN. 
Temporal dark noise can be estimated as CTdark.temp 

^ 1.6 DN and its uncertainness is 0.2 DN. 
Photo Response Non-Uniformity(PRNU) is less than 0.5% {aught.spat ~ 12 DN, mean value 



K 




(4) 



7 



of the signal is 2600) for all colour channels. Temporal light-depended noise can be estimated 

as aiight.temp ~ 14 DN. 

For further increasing of camera's dynamic range Spatially Varying pixel Exposures tech- 
nique can be used. As it shown above, in a quasimonochromatic light it is possible to receive 
around 7200 quantization levels of the input signal, which is a considerable improvement over 
3066 for the same image detector. 

Obtained results are an approximation of the camera's photo sensor characteristics be- 
cause of sufficient dispersion of noise characteristics between cameras of same model and 
presence of the noise-reduction on-chip circuitry. 

Application of described linearization method allows to increase the dynamic range of 
images produced by commercial digital photocamera. Prom carried out experiments it follows 
that inexpensive commercial photo cameras can be used in optical-digital imaging systems. 
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